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Method and apparatus lor segmenting dale to create mixed raster corilent planes 



(57) An improved technique tor compressing a color 
or gray scale pixel map representing a document using 
an MRC format ncludes a method of segmenling an 
original pixel map into two planes (12.16). andthencom- 
oressing the data or each plane In an efficient manner. 
The image is segmented by separating the mage into 



two portions at the edges. One plane contains image 
data lor the daiti sides ol the edges. whBe image data 
for the bright sides 61 the edges and *ie smooth portions 
ol the image are placed on the other plane. This results 
in improved image compression ratios and enhanced 
image quality. 
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Description 

[0001] This invention relates generally to irnage 
prcxessingand, more particularly, to techniques lor seg- 
menting, classifying and/or compressing the digital rep- s 
resentation o1 a ck)cumenl. 

[0002] Documents scarvied at high resolutions re- 
quire very Jarge amounts of storage space. Instead of 
being stored as is . the data is typically subjected to some 
form oi data compression in order to reduce its volume, 
and thereby avoid the high costs associated with storing 
it. "Lossless* conripression methods such as Lempel-Ziv 
Welch (LZW) do not perform particularly well on 
scanned pixel maps. While 'lossy* methods such as 
JPEG work fairly well on continuous-tone pixel maps, 
they donot work partkrularly well on the parts ol the page 
that contain text. To optimize image data compression, 
techniques, which can recognize the type of data being 
compressed, are needed. 

[0003] Known compression techniques are described 
In US-A-5778092, US-A-5251271, US-A-5060980. US- 
A-57e4l76. US-A-530331 3 and US-A-5432870, 
[0004] In one embodiment, the present invention die- 
ctoses e method of segmenting a pixel map represent 
tatk>n of a document which includes the steps of: acquir- 
ing a block of the digital image data, wherein the digital 
image data is composed of light intensity signals in dis- 
crete kx:atk>ns; designating a classificatk>n tor the block 
ar)d prdvkjing an indication about a context of the bk>ck: 
segmenting the light intensity signals in the bkxik into 30 
an upper subset and a k>wer subset based upon the des- 
ignated classlficatton; generating a selector set which 
tracks the fight intensity segmentation: and separately 
compressing the digital image data contained in the up- 
per and lower subsets. ^ 
[0005] In ar^other embodiment, the present inventk>n 
disckDses a method of classifying a block of digital image 
data into one of a plurality of image data types, wherein 
the block of data is composed of light intensity signals 
in discrete locations: whk:h includes: dividing the block 40 
into a bright region and a dark region; dividing a low pass 
filtered version of the block into a bright regk>n and a 
dark region; calculating average light intensity values for 
each of the bright region, the dark regkxi, the filtered 
bright region and the filtered dark region; and comparing ^5 
a difference between the bright region and the dark re- 
gion average light inienslty values to a f ihered difference 
between the bright regton and the dark region average 
filtered light intensity values; if the average light intensity 
difference and the average filtered light intensity differ- so 
ence are approximately equal finding a range of values 
in which the difference value falls, and classifying the 
block based upon the value range; and if the average 
light intensity difference and the average tittered light in- 
tensity difference are not approx innately equal finding a 55 
range of values in which the filtered difference value falls 
and classifying the block based upon the filtered value 
range. 



[0006] Some examples of methods apcording to the 
present invention will rK>w be described with reference 
to the acoonipanying drawings,* in which:- * 

Figure 1 illustrates a composite image btk! includes 
an example of how such an image may be decom- 
posed into three MRC image planes- an upper 
plane, a bwer plane, and a selector plane; 
Figure 2 contains a detailed view of a pixel map and 
the manner in which pixels are grouped to form 
bkx:k6; 

Figure 3 contains a flow chart .which illustrates gen- 
erally, the steps performed to practice the Invention; 
Figure 4 contains a detailed lllustratbn of the man- 
ner in which blocks may be classified according to 
the present inventbn; 

Figure 5 contains a detailed iilustiatkxi of the man- 
ner in which btocks may be segmented based upon 
their classificatkm according to the preserit Inven- 
tion; 

Figure 6 contains the details of one. emt)odiment of 
the manner in which block variatkxi can be meas- 
ured as required by the embodiment of the invention 
shown in Figure 4; 

Figure 7 contains the details of an ernbodiment of 
the inventkxi describing classlRcalbn of blocks 
based upon the block variation nneasurement pro- 
. vkjed in Figure 6; 

Figure 8 contains the details Of an ernbodiment of 
the invention for which context may be updated 
based upofi the bk)ck classification provided In Fig- 
ure 7; and. 

Figure 9 contains the details of another embodi- 
ment of the invention for updating context baaed up- 
on block classification as provkled In Figure 7. 

[0007] The present invention is directed to a method 
and apparatus lor separately processing the various 
types of data contained in a composKe image. While the 
invention will described In a Mixed Raster Content 
(MRC) technique, it may be adapted for use v^lth other 
methods and apparatus' and is not therefore, limited to 
a MRC format. The technique described herein is suit- 
able for use in vark>us devices required for storing or 
transmitting documents such as facsimile devices, im- 
age storage devices and the like, and processing of both 
color and grayscale black and white images are possi- 
ble. 

[0008] A pixel map is one In which each discrete lo- 
cation on the page contains a pcture element or *pb(el' 
that emits a light signal with a value that indicates the 
cok>r or, in the case of gray scale documents, how light 
or dark the image is at that location. As those skilled in 
the an will appreciate, most pixel maps have values that 
are taken from a set of discrete, non-negative integers. 
[0009] For example, in a pixel map for a cok^r docu- 
ment, individual separationG are often represented as 
digital values, often in the range 0 to 255. where 0 rep- 
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. rr.terant (i e. when CMYK separations are 
rdrormetweSv^^ 
! rJnre sebarations are used, and 255 represents 
chrominance 6epara^^<^ colorant or the highest value In 

T TrTa ^ayi«2le pixel mapthls typically trans- 

2S or^he Whitest tone possible. Thep»cel rr«ps <^ 
cScem n the currently prelerred embodiment 0I the 
concern in 1 representations ol 'scanned' im- 

?2^f;Jl2t^'ci 5 physical media using a digital scan- 
rCe^b1!^:apl/usedtomeanabina^pixe.map 

n«els can take one ol two values. 1 or 0. 
rooToi T rnnn<^ to the drawings .or a more de- 
!?«?Lscr Dtk?. 01 the MFIC lormat. pixel map 10 rep- 
Te^nt^g a cSS; or gray-scale document is pr^era^ 
decJi^pLd intoathree plane page lormat as mdcated 
^^.„r 1 Pixels on pixel map 10 are preferably 
"^rm bk^^ IB^S illustrated in Figure 2). to al- 
friof bet'eM^lge processing eTiciency. The doc j 

^LrJooTs^pa'T^tsed u'pon prc^eUned crHer^ 
J!! ex^ple. pixels that have values above a certam 
miv be placed on one plane, white those with 
m,ed^Wmayb«P^ to or below the threshold are 
values ^"^^T^ Seleclorplanel6keepstrack 
SrrP^Src:;?S;^ixelmaplSandma^a.l^ 
To an el^act spot on either upper plane 12 or tower plane 

?il11 The upper and lower planes are stored at the 
' KM riroih and number ol cotors as the original pixel 
^Tio bT^SiCrS .educed resolution. Selector 
"ZL^k fe crS?d and stored as a bitmap. It is impor- 
£^??o rtcogntee that while the terms 'upper' and 'low- 
er aTeTsS to ascribe the planes on which data re- 
:wes it is^n intended to limit the invention to any par- 
ticular arrangement or conliguration. 
M121 After processing, all three planes are com- 
L^Sd u^nga method suitable lor the type of date re- 
presseo usina , upper plane 1 2 and lower 

^fanl Tr^-^ c:^^^^ aJd s^^red us^g a lossy 
^ I^nrltl^ technique such as JPEG, while selector 
!Z reTcompessed and stored using a lossless 
^rpress-^ T-rSat such as gzip or CC1TT.G4. It would 
brapparent to one of skill in the art to comP'^ss ^n^ 

f.. l»«ri«d use of the output document. For example. 
irro^F:^imilearenr.9roup4 (MMR)^^^^^^ 
. erblbeusedlorseledorplanel6.s^^^^ 

rr^ton format used must be one of the approved 

?S^r";"J present invention digital image data is 
KablTprocised using a MRC technique such as 
SLscSed above. Pixel map lO represents a scanned 



image composed ol BflM intensity signals dispersed 
throughout the separation at discrete locations. Again, 
a light signal is emitted from each ol these discrete lo- 
cations, referred to as 'picture elements.' 'pixels or 
5 -pels ' at an intensily level which indicates the rnagrti- 
lude of the light bemg reflected Irorn the original »nage 
at the corresponding location in that separatioa 
100141 In typical MRC lashton. pixel map 10 must be 
panitiot^ed into two planes 12 and 14. Figure 3<»nlain6 
10 aschematic diagram, whfch outlines the overall process 
used to segment pixel map 10 into an upper plane 12 
and a lower plane 14 according to the present invention. 
Block 18 isacquiredas indicated in step 210: and is ctes- 
sified as indicated m step 220. In the preferred embqd- 
« iment ol the invention, block 1 8 will mltially be classified 
Z either UNlFORWt. SMOOTH. WEAK^EI^E pr 
?D6E. and its context - either. TEXT or- PICTURE - w.il 
be provWed. The block will then be reclassified as either 
SMOOTH or EDGE, depending upon the Initial classW- 
so cation and the context. Next, pixels IntokxA 18 are seg- 
mented - placed on either upper plane 12 or lower plane 
14 according to criteria that liniost appropriate lor the 
manner in which the block has been ctessffied ae indi- 
cated in step 230. Thfe process is repeated lor each 
2S block 18 in original pixel map 10 until the entire pixel 
map 10 has been proceseed; Upper plane 12. lower 
plane 14 and selector plane 16 are then separately com- 
pressed, using a technique that is most sultebte lor *ie 
type of data contained on each, as indicated in step 240. 
30 I001B1 Turing now to Figure 4. generally speakrig. 
classification of blocks 18 mtoorSe ol the lour categones 
in step 220 as described above is preferably completed 
in three steps. First, the variation ol pixel values wrthin 
the bkx:k is determined as indicated in step 310. Block 
55 variatkxi IS best determined by using statistical meas- 
ures which will be described in detail betow virilh refer- 
ence to Figure 6. Bkjcks with large vartetksnsthroughout 
are most likely to actually lie along edges of the image, 
while those containing little variations probably lie in uni- 
40 form or at least smooth areas. Measuring the venations 
within the block altows an initial classiTication to be as- 
signed to it 66 indicated in step 320. Next, wnage data 
whhin each bkx:k 18 is reviewed m detailto altow context 
Inlormatton <Le. whether the region is In the text or pc- 
4S ture region of the image) to be Updated and any neces- 
sary block re-classWicatlons to be performed as shown 
in step 330 The UNIFORM btocks are reclassified as 
SMOOTH, and the WEAK EDGE blocks are upgraded 
to EDGE in a TEXT context or rectessified as SMOOTH 
so in a PICTURE context. A smoothed version 20 of the 
image is also provided by applying a low pass filter to 
the pixel map 10. Smoothed image 20 is used in con- 
junction with original image date to oiler additional m- 
formation during classiTicafion. and also provides un- 
ss screened data for halftone regions. 

[0016] Figure 5 contains deteils ol the manner in 
which block 18 is segmented into two ptenes. as provid- 
ed in step 230 of Figure 3. The measurement begins by 
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first determining at step 410 whether the block being 
processed has inhially been classified as an EDGE in 
step 220. II so, the values Vp o1 each pixel in Ihe block 
are first compared to a brightness threshold value t^. 
v»;herein pixels that have values equal to or above t. are 
viewed as •bright' pixels, while those with values bek>w 
t are 'dark' pixels. Segmenting EDGE blocks simply 
includes placing dark pixels on upper plane 12 as indi- 
cated in step 440. and placing bright pixels on lower 
plane 14 as indicated in step 460. If it is determined at 
step 410 that block 18 Is not an EDGE, all pixels in the 
block are processed together, rather than on a pixel by 
pixel basis. Segmenting o1 SMCX)TH (non-EDGE) pix- 
els occurs as follows: H bkxk 1 8 is in the midst of a short 
run of bkDcks that have been classified as SMOOTH, 
and further, all blocks in this short mn are dark (Vp<t) - 
ail data m the block is placed on upper plane 12. II the 
entire block 1 8 is substantially snrKxrth (le. in a long run) 
or is bright (in a short run o! bright pixels), all data in 
block 18 is placed on tower plane 14. 
[0017] Turning now to Figure 6, the details oT one em- 
bodiment of the invention wherein initial bk)ck classifi- 
cation Via bk>ck variation measurement may be acconv 
plished as required by step 310 (Figure 4) are now de- 
scribed. A threshold, t^. whfch altows the block to be di- 
vided into two portions is first cateulated as indicated in 
step 510. In the preferred embodiment of the inventkxi. 
this threshold is obtained by performing a histogram 
analysis on the data in the block, but many standard 
methods can be used to perform this analysis. For ex- 
ample, the value that maximizes between distances of 
the criteria being used for separation or provides for 
maximum separation between the two portfons of the 
block can be selected. Those skilled in the art will rec- 
ognize that other methods of choosing the best Ihresh- 
okJ are available and the invention is not limited to this 
embodiment. Block 18 is then threshokJed into these 
two parts by comparing the light intensity value of each 
pixel to the selected threshold t^. as indicated in step 
520 As before. If the pixel value Vp is less than the 
threshokJ. the pixel is referred to as dark. If Vp is greater 
than or equal to t,. the pixel is bright. 
[001 8J As stated earlier, a smooth version 20 of the 
image is obtained by applying a low pass filter to the 
original image data. Average values lor bright and dark 
pixels are then obtained for both the original and 
smoothed sets of image data. Looking first at the bright 
pixels, one value cateulaied will be average 
value for all of the bright pixels in original pixel map 10 
(V 3 1^) which are tocated in the area covered by block 
IB as^indicated in step 540. Another value, Vbsmooth» 
the average value for all of the bright pixels in smoothed 
versk>n 20 of the image which are located in the area 
covered by btock 18 will also be obtained as «hown in 
step 560. Dark values are calculated similarly. That is, 
vqpixel. average value for all of the dark pixels in 
original pixel map 10 (Vp < t^) which are located in the 
area covered by block 18 will be obtained as shown in 



step 550. and \/oshnooiH^ average value for all of the 
dark pixels in the smoothed version 20 dl the image 
which are kx^ted m the area covered by bkx:k 18 will 
be obtained as in step 570. Once these average values 

s are obtained, the distances d and d^ between brighter 
and darker averages for pixel map.10 and smoothed Im- 
age 20 respectively are calculated as ind'cated in etep 
580. That is d= Vbpixel' Vdpixel' a^d d, = Vbswooth - 
V0SMOOTH Since d/d^ is typk:ally almost equal to 1 lor 

10 contone images, the ratfo of d/d, may be used to detect 
halftones. 

[0019] Figure 7 contains a detailed illustration of step 
320. of Figure 4, the preferred embodimerU of a process 
for initially classifying blocks 18. As shown, a relative 

IS comparison between d and d^ is obtained as indicated 
in step 61 0 in order to determine whether the bkx^k con- 
tains contone (d » d^) or halftorie data. Block 1B will vi- 
nially be classified as one of four types: UNIFORM. 
SMOOTH. WEAK EDGE or EDGE according to the 

so magnitude of the distance d or d,. Distance d is used to 
classify contone bkx:ks. while distance is used for 
halftones. For contone data d. the value from pixel map 
10» is compared to value Xq as shown in step 620. 
[0020] If d is very k>w (i.e. d< x^), all pixel values In 
the block are substantially the same and the block is 
classified as UNIFORM at step 640. II there are some- 
what smalt differences in pixel values In the block such 
that Xo<d<x^ as shown in step 622. the block is classified 
as SMOOTH, at step 650. If there are fairly large differ- 

30 ences in pixel values in the block and x^<d<X2 al step 
624. the bkick will be classified as VVEAK EDGE. 11 the 
differences \n the bk)ck are very large and at step 
624, the bkx:k will be classified as an EDGE at step 670. 
[0021] If d/d|, IS not apprc»(imately equal to 1 , is 

3S compared to threshold yo at step 630. it should be noted 
there that two different sets of thresholds are applied for 
halftones and contones. Thus, on most occasions. 
Xo'yo. Xi''y^. and X2''y2. The process used to classify 
halftone bkx:ks is similar to that used for cootone data. 

40 Thus. If d^cyo at step 630 the block is classified as UNI* 
FORM at step 640. If yo<ds<yi s^ep B32, the block is 
classified as SMOOTH, at step 650. If yi<d,<y2 as indi- 
cated in step 634. the block is classified as a WEAK 
EDGE at step 660. If d^2 ^^^P tflock will be 

45 classified as an edge at step 670. 

[0022] Referring now to Figures B and 9. the details 
for updating the context o1 the block will now be provid- 
ed. The context of a bkx;k is useful when the average 
between the dark and bright areas of the block is rela- 
te tively high. When this is the case, the block can classi- 
fied as an EDGE as long as its context is TEXT. The 
context is inhially set equal to PICTURE. It is changed 
to TEXT if one of two rules is satisfied: (1) the block be- 
ing processed is in a long run of UNIFORM blocks and 

ss the average of the dark pixel values in the block is great- 
er than a preset brightness threshoU; or (2) the block 
has been classified as either UNIFORM. WEAK EDGE, 
or EDGE, one of the top. left or right neighboring blocks 
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has a context which has been set equal to TEXT, and 
the diflerence between that neighboring block and the 
current block is smaller than a preset propagation 

loST^ *i^st to Figure 8. determining whether 

block context should be changed according to the first 
rule requires finding a run of bk)cks that have been clas- 
sitied as UNIFORM as Indicated in step 704, Finding a 
run of UNIFORM blocks typically involves companng 
me number o1 consecutive UNIFORM blocks to a run 
. length threshold t^u as indicated in step 706. The run 
length threshold sets the number o1 consecutive blocks 
that must be classified as UNIFORM lor a run to be es- 
tablished. As also indfcated in step 706, Vdpixel. a^" 
eraoe value o1 the dark pixels lor consecutive bkDcks is 
cornpared to the brightness threshold t^ A large number 
o1 consecutive UNIFORM blocks with high brightness 
levels usually indicates that the blocks contain large 
background page areas (I.e. large white areas), thereby 
indicaling thai lexl is present. Thus, if the number of con- 
secutive UNIFORM blocks exceeds t^u and VoppcEL > 
ig. the context lor the block is changed to TEXT as indh 
ceted m step 708. • 
100241 If either the number of identified consecutive 
blocks is too small to establish a run or the blocks are 
dark (VoPixEL C the context will remain set equal to 
PICTURE Whether additional runs are present m the 
block w'lll be determined as indicated in step 710, and if 
so the process will be repeated as indicated in the illus- 

iSSl Turning now to Figure 9. changing the context 
o1 a block to TEXT under the second rule first requires 
providing a propagation threshold The propagation 
threshold def ines the level ot brightness that will indicate 
that the bkx:k covers blank page areas. Under the sec- 
ond mie, the context win be changed Irom pkrture to text 
at step B08 It the block is not SMOOTH (l.e. is UNIr 
FROM and EDGE or a WEAK EDGE) as shown in step 
802 either its top, left or right neighbor has a text context 
as indicated in step 804 and vbdif. the average diHer- 
ence between bright pixels in the block and bright pixels 
in the neighbor text context block is less than t^ as 
shown in step 806. Neighbor blocks are checked be- 
cause presumably bkx:ks that contain text vyrill be locat- 
ed next to other bkxks that contain text. However, the 
brightness value ot the block is compared to that o1 rts 
neighbor to assure that this is the case. In other words, 
even il Ihe bkxk has a neighboring block with a lext con- 
text a large diflerence between the average brightness 
ot bk>ck and its neighbor means that the block contain 
cJoes not contain the large blank page areas that indicate 
the presence ot text. 

10026] Again, the present invention is directed to seg- 
menting the data by first identifying blocks that contain 
the edges of the image and then separating the bkxrks 
such that those which contain the smooth data and 
bright sides of the edges are placed on the lower plane 
and the dark sides of the edges are placed on the upper 



plane. Once each of the respective planes fe generated, 
ordinary MRC processing continues. That is.each plane 
is compressed using an appropriate compression tech- 
nique. In the currently preferred enrtbodiment. upper 

5 plane ^2 and tower plane 14 are compressed using 
JPEG while the selector plane 16 is compressed uising 
a symbol based pattern matching technique such as 
CCITT Group IV or a method of classifying scanned 
symbols into equivalence classes such as that de- 

10 scribed in US-A 5,778.095 to Davies issued July 7. 
1998, the contents of which are hereby incorporated by 
reference. The planes are then joined together and 
transmitted to an output device, such asa facsimile ma- 
chine or storage device. 

IS 

Claims 

1, A method of segmenting digital irrage data lor 
20 mixed raster content processing, comprising: 

a) acquiring a block of the digital image data, 
wherein the digital image data is composed ol 
light intensity signals in discrete locations; 
2S b) designating a claes'incation for said block 

and providing an indk:ation about a context d 
saidbtock; 

c) segmenting said light intensity signals in saki 
bkx:k Into an upper subset arKf a k>wer subset 

30 based upon said deslgrtated classification; 

d) generatng a selector set which tracks said 
light intensity segmentation; and 

e) separately compressing the digital image da- 
ta contained in said upper and tower subsets. 
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2. A method of segmenting digital image data as 
claimed in claim 1 . wherein said classification indi- 
cates that said btock contains substantially smooth 
data and/or substantially edge data. 

3, A method of segmenting digital image data as 
claimed in claim 1 or claim 2. wherein said classm- 
cation data designating step further comprises: 

a) measuring an amount of light intensity sigriat 
variation throughout said block; 

b) assigning a classification to said block based 
upon said measured light intensity signal vari- 
ation; and 

c) updating said context indtoation for sakJ 
btock, and designating classification for said 
btock based upon said updated context. 

A method of segmenting digital image data as 
claimed in any of the preceding claims, further com- 
prising: 

a) dividing a tow pass filtered version of said 
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block into a bright region and a dark region; 

b) calcuteting average fihered light intensity val- 
ues lor said bright region and lor said-dark re- 
gion; and 

c) obtaining a ditterence in average filtered light 
intensity values between said bright region and 
said dark region. 

A method ot segmenting a block ot digital image da- 
ta into an upper and iower subset, wherein the block 
dl data is composed of light intensity signals in dis- 
crete locations, comprising: 

a) determining v\fhether the block is kjcated on 
an edge in the digital image; 

b) H the bkxk is on an edge, comparing a mag- 
nitude of each light intensity signal in the block 
to a brightness threshold and placing said sig- 
nal in the upper subset If said light intensity 
magnilude exceeds said brightness threshold 
or in the lower subset If sakJ ii^t intensity mag- 
nitude IS less than sakJ brightness threshold; 
and 

c) if the block is not located on an edge, placing 
the block in the upper subset it the btock is in a 
group of bkxjks that have fight intensity values 
which are indicative of 6nrK»th and dark image 
data, and othervnse placing the block in the low- 
er subset. 

A method of classifying a block of digftal image data 
into one of a plurality of image data types, wherein 
the block of data is composed of light intensity sig- 
nals in discrete locations, comprising: 

a) dividing the bkXJk into a bright regk>n and a 
dark region; 

b) dividing a low pass filtered versk>n of said 
block into a bright region and a dark region; 

c) calculating average light intensity values for 
each of said bright region, saxj dark region, 
said filtered bright region and said filtered dark 
region; and 

d) comparing a dillerence between said bright 
regk>n and said dark region average light inten- 
sity values to a filtered difference between said 
bright region and said dark region average fil- 
tered light intensity values; 

e) if said average light intensity difference and 
said average filtered light intensity difference 
are approximately equal finding a range of val^ 
ues in which sakj difference value falls, and 
classifying said block based upon sakJ value 
range; and 

f) if said average light intensity difference and 
said average filtered light intensity difference 
are not approximately equal finding a range of 
values in which said filtered difference value 



falls and classifying said block based upon said 
filtered value range. 

A method according to any of claims 1 to 4, wherein 
blocks are classified by a method according to claim 
5 or claim 6. 
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